Convolutional Neural Network Bootstrapped by Dynamic Segmentation and Stigmergy-Based Encoding for Real-Time Human Activity Recognition in Smart Homes.

Recently, deep learning (DL) approaches have been extensively employed to recognize human activities in smart buildings, which greatly broaden the scope of applications in this field. Convolutional neural networks (CNN), well known for feature extraction and activity classification, have been applied for estimating human activities. However, most CNN-based techniques usually focus on divided sequences associated to activities, since many real-world employments require information about human activities in real time. In this work, an online human activity recognition (HAR) framework on streaming sensor is proposed. The methodology incorporates real-time dynamic segmentation, stigmergy-based encoding, and classification with a CNN2D. Dynamic segmentation decides if two succeeding events belong to the same activity segment or not. Then, because a CNN2D requires a multi-dimensional format in input, stigmergic track encoding is adopted to build encoded features in a multi-dimensional format. It adopts the directed weighted network (DWN) that takes into account the human spatio-temporal tracks with a requirement of overlapping activities. It represents a matrix that describes an activity segment. Once the DWN for each activity segment is determined, a CNN2D with a DWN in input is adopted to classify activities. The proposed approach is applied to a real case study: the "Aruba" dataset from the CASAS database.

Dynamic Segmentation of Sensor Events for Real-Time Human Activity Recognition in a Smart Home Context.

Human activity recognition (HAR) is fundamental to many services in smart buildings. However, providing sufficiently robust activity recognition systems that could be confidently deployed in an ordinary real environment remains a major challenge. Much of the research done in this area has mainly focused on recognition through pre-segmented sensor data. In this paper, real-time human activity recognition based on streaming sensors is investigated. The proposed methodology incorporates dynamic event windowing based on spatio-temporal correlation and the knowledge of activity trigger sensor to recognize activities and record new events. The objective is to determine whether the last event that just happened belongs to the current activity, or if it is the sign of the start of a new activity. For this, we consider the correlation between sensors in view of what can be seen in the history of past events. The proposed algorithm contains three steps: verification of sensor correlation (SC), verification of temporal correlation (TC), and determination of the activity triggering the sensor. The proposed approach is applied to a real case study: the "Aruba" dataset from the CASAS database. F1 score is used to assess the quality of the segmentation. The results show that the proposed approach segments several activities (sleeping, bed to toilet, meal preparation, eating, housekeeping, working, entering home, and leaving home) with an F1 score of 0.63-0.99.